The supplementary materials are organized as follows. In Appendix B, we discuss the motivations for our UniHOI. In Appendix C, we provide an in-depth explanation of differences between our UniHOI and previous CLIP-based methods. In Appendix D, we examine the effects of VL foundation models of different scales. In Appendix E, we provide an detailed explanation of the training and hyperparameter setting.

The final experiments have confirmed the validity of these perspectives.

Human-object interaction (HOI) detection aims to comprehend the intricate relationships between humans and objects, predicting triplets, and serving as the foundation for numerous computer vision tasks. The complexity and diversity of human-object interactions in the real world, however, pose significant challenges for both annotation and recognition, particularly in recognizing interactions within an open world context. This study explores the universal interaction recognition in an open-world setting through the use of Vision-Language (VL) foundation models and large language models (LLMs). The proposed method is dubbed as UniHOI. We conduct a deep analysis of the three hierarchical features inherent in visual HOI detectors and propose a method for high-level relation extraction aimed at VL foundation models, which we call HO prompt-based learning.